Skip to content

perf: profile guided optimizations#1126

Open
henderkes wants to merge 30 commits intomainfrom
feat/pgo
Open

perf: profile guided optimizations#1126
henderkes wants to merge 30 commits intomainfrom
feat/pgo

Conversation

@henderkes
Copy link
Copy Markdown
Collaborator

@henderkes henderkes commented Apr 30, 2026

What does this PR do?

Disclaimer: zig/clang ONLY!

  • reworks Zig patches to happen on zig-cc instead of patching shared extensions makefiles
  • instead of looking for shared runtime begin and end objects on the host machine, compile them from llvm source - this fixes also libraries if we decide to build them as shared libraries instead of static, useful for v3

adds profile guided optimizations to the project

  • closely aligned to windows' --enable-pgi -> --with-pgo workflow
  • function level profiling and value profiling
  • cs profiling for an optional second pgo pass
  • full and thin lto compatibility, required a few library patches
  • coincidentally figured out that our cmake $AR set caused issues

not so nice:

  • need to patch because Go doesn't call libc's atexit handlers that are usually responsible for writing out profile data
  • -fprofile-continuous works without it, but can't do vp and also confuses older llvm-profdata binaries...
  • need a llvm toolchain on the computer... older version is fine, but I'm not sure how far back. 19+ work for zigs llvm 21/22 (dev).

also adds required pre-work for: php/frankenphp#2361

  • pgo for frankenphp (just add SPC_CMD_VAR_FRANKENPHP_XCADDY_MODULES=... ${FRANKENPHP_SOURCE_DIR} and rewrite the env var to be set before the second GlobalEnvManager pass)

Little preview of frankenphp benchmarks:

❯❯ frankenphp git:(perf-tests) 23:04 LD_LIBRARY_PATH=/home/m/static-php-cli/pgobuild FRANKENPHP_BIN=/home/m/static-php-cli/pgobuild/frankenphp ./profiles/benchmark.sh
/home/m/static-php-cli/pgobuild/frankenphp
script                      req/s        avg        p50        p99
cookies_headers             36044     7.42ms     6.40ms    25.15ms
file_io                   7568.43    33.41ms    32.67ms    43.67ms
frankenphp_log            38847.7     6.84ms     6.00ms    21.43ms
helloworld                42319.7     6.23ms     5.49ms    19.86ms
large_response               9931    21.11ms    18.06ms    61.10ms
mandelbrot                1437.94   173.65ms   164.08ms   239.88ms
post_body                 36753.9     7.18ms     6.32ms    22.20ms
random                    13163.2    19.65ms    17.58ms    60.17ms
status_codes              35443.7     7.44ms     6.55ms    22.96ms
streaming                 19274.1    13.32ms    12.32ms    35.60ms
total req/s                240784
❯❯ frankenphp git:(perf-tests) 23:06 ./profiles/benchmark.sh
/home/m/frankenphp/caddy/frankenphp/frankenphp
script                      req/s        avg        p50        p99
cookies_headers             27353     9.32ms     8.84ms    16.48ms
file_io                   5483.86    45.96ms    43.31ms    74.65ms
frankenphp_log            26572.2     9.52ms     9.07ms    17.25ms
helloworld                27481.2     9.20ms     8.74ms    17.74ms
large_response            5292.43    47.75ms    45.78ms    73.83ms
mandelbrot                2188.24   113.81ms   101.98ms   187.42ms
post_body                 26340.5     9.59ms     9.16ms    18.00ms
random                     2754.5    91.28ms    89.72ms   118.68ms
status_codes              26327.5     9.62ms     9.13ms    19.67ms
streaming                 2224.59   113.31ms   110.16ms   153.59ms
total req/s                152019

Little preview of php-cli benchmarks:

❯❯ frankenphp git:(perf-tests)  19:30 yes n | PHP_BIN=/home/static-php-cli/buildroot/bin/php phoronix-test-suite benchmark phpbench
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 2 Minutes [12:32 UTC]
        Started Run 1 @ 12:31:21
        Started Run 2 @ 12:31:39
        Started Run 3 @ 12:31:54

    [8192] Using null as an array offset is deprecated, use an empty string instead in pts_strings:586

    PHP Benchmark Suite:
        1892968
        1903458
        1923582

    Average: 1906669 Score
    Deviation: 0.82%
❯❯ frankenphp git:(perf-tests)  19:32 yes n | PHP_BIN=/usr/bin/php-zts phoronix-test-suite benchmark phpbench
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 2 Minutes [12:34 UTC]
        Started Run 1 @ 12:32:57
        Started Run 2 @ 12:33:18
        Started Run 3 @ 12:33:34

    [8192] Using null as an array offset is deprecated, use an empty string instead in pts_strings:586

    PHP Benchmark Suite:
        1646683
        1646953
        1635145

    Average: 1642927 Score
    Deviation: 0.41%
❯❯ frankenphp git:(perf-tests) 19:44 yes n | PHP_BIN=/usr/bin/php phoronix-test-suite benchmark phpbench
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 2 Minutes [12:46 UTC]
        Started Run 1 @ 12:44:49
        Started Run 2 @ 12:45:13
        Started Run 3 @ 12:45:32

    [8192] Using null as an array offset is deprecated, use an empty string instead in pts_strings:586

    PHP Benchmark Suite:
        1340276
        1323430
        1315188

    Average: 1326298 Score
    Deviation: 0.96%

@henderkes henderkes requested a review from crazywhalecc April 30, 2026 12:18
@henderkes henderkes changed the title Feat/pgo perf: profile guided optimizations Apr 30, 2026
@henderkes
Copy link
Copy Markdown
Collaborator Author

Symfony gains are also good, but strongly depend on the route. Something like the homepage (no session, no nothing, just a ton of file accesses) sees rather small gains, while others see decent gains. Training data was on all routes of the project, but benchmark only on few. @nicolas-grekas

The smaller the route surface of the project and the more time it spends in php, the more likely we are to see higher gains.

❯❯ static-php-cli git:(feat/pgo) 17:30 GOGC=1000 FRANKENPHP_BIN=/usr/bin/frankenphp BENCH_SEC=300 /home/m/symfony_demo/quick-bench.sh 
BIN=/usr/bin/frankenphp  PHPRC=/home/m/symfony_demo/var/php-ini
route                                                     req/s        avg        p50        p99
/en/                                                    4415.08    14.49ms    14.40ms    17.30ms
/en/blog/                                               1640.11    39.01ms    38.85ms    44.12ms
/en/login                                               1480.98    43.98ms    39.84ms    58.49ms
/en/blog/posts/lorem-ipsum-dolor-sit-amet-consectetur-adipiscing-elit      1364.28    46.89ms    46.81ms    56.31ms
TOTAL                                                   8900.45
❯❯ static-php-cli git:(feat/pgo) 17:55 GOGC=1000 FRANKENPHP_BIN=/home/m/static-php-cli/buildroot/bin/frankenphp BENCH_SEC=300 /home/m/symfony_demo/quick-bench.sh 
BIN=/home/m/static-php-cli/buildroot/bin/frankenphp  PHPRC=/home/m/symfony_demo/var/php-ini
route                                                     req/s        avg        p50        p99
/en/                                                    4560.36    13.96ms    13.85ms    16.73ms
/en/blog/                                               1739.43    36.56ms    36.26ms    42.42ms
/en/login                                                  2467    30.66ms    23.94ms   192.95ms
/en/blog/posts/lorem-ipsum-dolor-sit-amet-consectetur-adipiscing-elit      1860.54    34.12ms    34.10ms    38.23ms
TOTAL                                                   10627.3

@henderkes henderkes marked this pull request as draft May 5, 2026 20:15
@henderkes
Copy link
Copy Markdown
Collaborator Author

Sorry that I still opened this against v2, I wasn't quite ready to switch to v3 yet for our packaging. It's my last v2 PR though, I promise! Updated description to explain all this does.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know much about frankenphp's source code. Is it possible that this is implemented in frankenphp instead of in spc's patch?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dunglas thoughts? We could guard it behind a -DPGI_FLUSH_PATCH define?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not, but IMHO this should be fixed right in Go

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean invoking libc atexit handlers if exiting when linked with CGO_ENABLED=1? I've not looked at Go source code yet, so if you could take that over I'd be grateful.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I'll take a look but no ETA

Comment on lines +60 to +64
// OpenSSL's Configure ignores env CFLAGS for its target template; pass our flags as extra args after the target.
$userCFlags = trim((string) getenv('SPC_DEFAULT_C_FLAGS'));
$userLdFlags = trim((string) getenv('SPC_DEFAULT_LD_FLAGS'));
$userExtraFlags = trim($userCFlags . ' ' . $userLdFlags);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be only for PGO though, I think. Openssl has -O3 optimization by default, and our SPC's default is -Os. This may reduce the performance of OpenSSL.

Considering the special nature of OpenSSL, I think it's better to handle this separately.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be only for PGO though, I think. Openssl has -O3 optimization by default, and our SPC's default is -Os. This may reduce the performance of OpenSSL.

We switched the default to O3, but regardless of that, I think all libraries should use the user set CFLAGS.

Comment thread src/SPC/builder/unix/library/bzip2.php Outdated
Comment on lines +23 to +26
$this->addOption('pgi', null, null, 'Forward --pgi to the inner build (instrumented binaries).');
$this->addOption('cs-pgi', null, null, 'Forward --cs-pgi to the inner build (cs-instrumented binaries).');
$this->addOption('pgo', null, null, 'Forward --pgo to the inner build (use collected profile data).');
$this->addOption('libs-only', null, null, 'Build only the libraries needed by the configured exts (skip PHP and extension build).');
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have options in build command and can pass in craft.yml, why we need to add in craft command separately?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait we can? I didn't see that!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@crazywhalecc I can't find it

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (isset($craft['build-options'])) {
if (!is_assoc_array($craft['build-options'])) {
throw new ValidationException('Craft file build-options must be an object');
}
foreach ($craft['build-options'] as $key => $value) {
if (!isset($build_options[$key])) {
throw new ValidationException("Craft file build-options {$key} is invalid");
}
// check an array
if ($build_options[$key]->isArray() && !is_array($value)) {
throw new ValidationException("Craft file build-options {$key} must be an array");
}
}
} else {
$craft['build-options'] = [];
}

Just use build-options like this:

extensions: "..."
sapi: cli
build-options:
  enable-zts: true
  pgi: true
  pgo: true
  ...

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I thought you meant the libs-only option.

So for pgi/pgo[/cs-pgi/pgo] is because they need to be called in that order. Constantly changing the .craft file in between is bad DX.

if ($retcode !== 0) {
$this->output->writeln('<error>craft build failed</error>');
return static::FAILURE;
if ($this->getOption('libs-only')) {
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since craft command is designed for building static php binaries easily, we are not defining any other targets and libs build in craft though.

I did consider implementing other building goals in v3, or building more flexible libraries that I wanted, rather than just static PHP or extensions, but I think that needs to be discussed.

But since this branch is for v2, I don't have strong objections.

@henderkes henderkes marked this pull request as ready for review May 6, 2026 10:55
Comment thread src/SPC/builder/unix/library/bzip2.php Outdated
}
$this->buildEmbed();
}],
// frankenphp doesn't rebuild php-src; xcaddy links against the deployed libphp.so
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v3 has --maintainer-skip-build option here (just a reminder)

Comment on lines +25 to +32
$stripLto = static fn ($s) => preg_replace('/(^|\s)-flto(=\S+)?(?=\s|$)/', ' ', (string) $s);
$origC = $this->builder->arch_c_flags;
$origCxx = $this->builder->arch_cxx_flags;
$origLd = $this->builder->arch_ld_flags;
$this->builder->arch_c_flags = clean_spaces($stripLto($origC));
$this->builder->arch_cxx_flags = clean_spaces($stripLto($origCxx));
$this->builder->arch_ld_flags = clean_spaces($stripLto($origLd));
$make = UnixAutoconfExecutor::create($this)
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead I think we could add $executor->getEnv() and replace temporarily with $executor->setEnv().

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The responsibilities of PgoManager are not clearly defined, but I can't think of a better way to handle v2. Perhaps v3 can be implemented more elegantly through conditional annotations and hooks.

@crazywhalecc
Copy link
Copy Markdown
Owner

After review, I have a strong feeling that this would be much smoother to implement on v3.

Use a separate PgoContext to declare the process, use attributes to define and hook the build phases, and use a separate new command BuildTargetPgoCommand to handle the entire split process.

@henderkes
Copy link
Copy Markdown
Collaborator Author

Yes, I agree, it's just faster for me to iterate on the v2 branch still due to familiarity. If you prefer, I can rework this on top of v3, but it will likely take me a week or two.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants